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Abstract 

The present research investigates 
characteristics exhibited in pilot convective 
weather decision making in en route airspace. 
In a part-task study, pilots performed weather 
avoidance under various encounter scenarios. 
Results showed that the margins of safety that 
pilots maintain from storms are as fluid as 
deviation decisions themselves. 

1 Introduction 

Weather related disruptions account for about 
seventy percent of the delays in the U.S. 
National Airspace System (NAS) [1], The 
presence of convective weather hazards reduces 
traffic flows in impacted airspace and forces 
planes to deviate from planned trajectories. In 
today’s operation, strategic weather avoidance 
reroutes (defined here as 2-6 hours time frame) 
are generated based on weather forecasts 
provided by the Collaborative Convective 
Forecast Product (CCFP) and implemented at 
the FAA Air Traffic Control System Command 
Center (ATCSCC) according to the National 
Severe Weather Playbook (NSWP) [2]. Tactical 
weather avoidance reroutes (defined here as 0-2 
hours time frame) are generated by pilots in the 
air and submitted as requests to air traffic 
controllers, who vet the requested reroutes 
against conflicts with surrounding traffic. 
Reroutes free of conflicts can be readily 
approved and implemented; those that create 
potential conflicts will need to be modified 
through further negotiations between the pilot 


and controller. Handling pilot requests for 
deviations in the event of thunderstorms further 
burdens controllers at a time when they are 
already confronted with increased workload 
owing to increased traffic density and 
complexity created by constricted operable 
airspace [3]. As controller workload is a major 
determinant of airspace capacity, how to ease 
the demand on controllers when managing 
traffic through thunderstorms is a pressing issue 
in realizing the visions of the Next Generation 
Air Transportation System (NextGen). 

Like most recent advances in airspace 
operations, solutions to reducing controller 
workload in managing traffic through 
thunderstorms involve automation. Specifically, 
research has focused on developing automated 
algorithms for generating conflict-free reroutes 
[4-8] around identified hazardous regions [9- 
13]. While the generated reroutes meet the 
requirement from a traffic management 
standpoint, they do not necessarily reflect how 
pilots deviate for weather. As such, further 
negotiations likely will still be needed before 
reaching resolutions satisfying both the pilot 
and controller. To date there has been little 
research examining the strategic aspects of pilot 
weather avoidance in en route airspace. The 
present research aims to shed some light on the 
process. 

1,1 Automated Weather Avoidance for 
Traffic Management 

While pilots bear the responsibility for 
deviating around hazardous weather, controllers 
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oversee the weather avoidance decisions made 
by individual pilots to maintain separation 
assurance and ensure efficient use of available 
airports, airspace, and other resources as part of 
their Traffic Flow Management (TFM) 
responsibilities [14]. Various automated 
algorithms for weather avoidance have been 
developed under these premises for controllers 
[4-8]. The algorithms reroute aircraft around 
weather by first identifying whether their 
planned trajectories penetrate storm cells, which 
are typically represented computationally as 
polygons. If penetration occurs, the algorithms 
search for the most efficient reroutes around the 
storm cells. Refinements to the algorithms have 
included taking into account the unreliability of 
weather forecasts and periodically issuing new 
routes based on updated forecasts [4] and 
allowing users to add custom constraints, or 
wiggle room, to the reroutes [7]. 

Other refinements to automated weather 
avoidance have involved developing more 
elaborated descriptions of the obstructing storm 
cells. For example, Avijan and colleagues 
attempted to describe weather hazards as 4D 
grid-based weather avoidance hazard volumes 
instead of polygons [9-11]. The 4D description 
acknowledges the dynamic and time-variant 
nature of weather phenomena while data coded 
in a grid system make them more conducive to 
be manipulated mathematically. 

DeLaura and colleagues took a further step 
beyond describing storm impacted regions 
based on the attributes of the weather activities 
by taking into account how pilots behaved when 
they were confronted with storms of similar 
attributes in the past [12], [13]. They analyzed 
real aircraft convective weather encounters and 
identified meteorological factors that best 
predicted pilots’ decisions given specific 
weather conditions. The end result of their effort 
is the Convective Weather Avoidance Model 
(CWAM), a statistical model of pilot weather 
avoidance behavior in en route airspace. 
Outputs of the CWAM models specify impacted 
regions that pilots deviate to avoid and the 
associated probabilities (e.g. regions that 90%, 
80%, etc. will avoid). Validations of CWAM 
model predictions of deviations and non- 


deviations against observed ones showed that 
prediction errors are around 25% [15-17]. 

1.2 Pilot Weather Avoidance Decisions: 
Situated and Opportunistic 

The effectiveness of most of the aforementioned 
algorithms has only been demonstrated in fast- 
time simulations. In the few instances where 
comparisons were made between algorithm 
decisions and pilot decisions based on actual 
weather data, pilot decisions were found to 
match algorithm decisions to a large extent but 
differ in small and interesting ways. In 
comparing actual pilot decisions with CWAM 
outputs, Chan and colleagues found 3% of 
instances where pilots stayed on their flight 
plans without encountering a CWAM polygon 
as would be predicted by CWAM outputs [18], 
Chan and colleagues speculated that those pilots 
waited until getting closer to the storms to 
determine whether the flight paths were 
acceptable. In addition, pilots were also 
frequently found to skirt or clip the edges of the 
CWAM polygons. 

Discrepancies between CWAM outputs 
and actual pilot behaviors were also observed by 
DeLaura and Evans, who noted a case where the 
pilot made a large deviation around a region of 
weather whose intensity would be considered 
benign and thus penetrable according to CWAM 
predictions [12], Crowe, DeLaura, and 
Matthews speculate that the discrepancies could 
be due to differences in weather information 
available to the flight deck and to ground-based 
CWAM [17]. Information discrepancies 
notwithstanding, Crowe et al. acknowledge that 
some deviation decisions are likely strategic, 
reflecting influences from opportunities present 
in a particular encounter; for instance, pilots 
may trade off comfort for efficiency, or vice 
versa, under different circumstances. 

1.3 Present Research 

The present research aims to systematically 
examine the strategic aspect of pilot weather 
decision making in the tactical time frame (0-2 
hours) in en route airspace. In a part-task study, 
we presented pilots with weather encounter 
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scenarios that varied in distance to encounter 
and point of approach, with the goal of 
manipulating urgency and opportunity, 
respectively. Predicted weather forecasts up to 
40 minutes were provided; the prediction was 
assumed to be 100% accurate. Pilots were asked 
to find a safe and efficient route around weather 
within the shortest amount of time using the 
provided onboard tools to evaluate and 
implement alternative trajectories. We evaluated 
the type of maneuvers pilots took to deviate for 
weather and the safety margins they maintained 
from storms. 

2 Method 

2.1 Participants 

Eighteen transport pilots with 1000 to over 5000 
hours of commercial flight experience 
participated in the study and were compensated 
$25/hr. The majority of them had experience 
using the Cockpit Situation Display (CSD) from 
participating in previous studies in the lab but 
none had experience with the predicted weather 
display interfaces. 

2.2 Apparatus 

The study was conducted using an IBM- 
compatible personal computer (PC) equipped 
with a 30” LCD display. Pilots manipulated the 
CSD using a computer mouse. 

The platform on which nowcast and 
forecasted predicted weather information was 
presented was the CSD, an extension of a 
Cockpit Display of Traffic Information (CDTI) 
(Fig. 1). The CSD is an interactive display 
prototype that has been in development in the 
Flight Deck Display Research Laboratory at 
NASA Ames Research Center for over a 
decade, for the purpose of implementing and 
testing future aviation concepts. The CSD 
supports both traditional 2D and advanced 3D 
perspective visualization models, and depicts 
the 4D interrelationship of traffic, terrain, and 
weather using a cylindrical volume metaphor. 

As part of a capability supporting 
trajectory -based operations (TBO), the CSD 


also includes the Route Assessment Tool 



Fig. 1. A screenshot of the CSD and RAT. The 
weather scenario shown here is the Middle type. 


(RAT), which integrates with the aircraft’s 
Flight Management System (FMS) and allows 
for in-flight trajectory replanning. A standard 
computer mouse serves as the input device for 
use with the CSD. The RAT adopts the principle 
of a direct manipulation interface [19-22] 
providing the functionality to create and 
visualize in-flight route modifications, downlink 
proposed route modifications to Air Traffic 
Control (ATC), receive route modifications 
from ATC, and execute modifications. The 
RAT supports the addition of waypoints at 
arbitrary latitudes- longitudes, and deletion of 
waypoints, through both clicking and dragging- 
and-dropping mouse operations. For each 
waypoint, pilots can also adjust an associated 
flight altitude and speed, thus enabling 4D 
trajectory in-flight planning. 

2.3 Design 

On each trial, pilots were presented with a 
weather encounter scenario in en route 
environments and asked to modify the planned 
trajectory if they found it unsafe according to 
predicted weather forecasts. We manipulated the 
distance between the starting location of 
ownship to the location on the initial trajectory 
where ownship was expected to reach the 


3 


WU, GOODING, SHELLEY, DUONG, & JOHNSON 


closest point of approach (CPA) to weather. 
These distances could be either 40 nm (~5 min) 
or 80 nm (~10 min). At either of the distances, 
ownship could encounter hazardous weather in 
one of four ways: 

• Middle : On these trials, the existing 3D 
ownship trajectory, and the 4D forecast 
ownship trajectory, penetrated one of the 
storm cells near the center of a storm 
front. It was designed such that it would 
be very inefficient to take a large detour 
and bypass all of the storm cells so that 
pilots would be more tempted to find an 
alternative route through the gaps 
between storm cells. 

• Initially clearing gap : On these trials, the 
existing 3D ownship trajectory was 
initially clear of the given line of storm 
cells, passing through a gap in the 
current weather depiction. However the 
forecast 4D trajectory was predicted to 
penetrate the storm cells. 

• Initially clearing edge : On these trials, 
the existing 3D ownship trajectory 
appeared to clear the leading or trailing 
edge of a line of storm cells in current 
weather depictions, but the forecast 4D 
trajectory was predicted to penetrate the 
storm cells. 

• Clear later : On these trials, the existing 
3D ownship trajectory penetrated storm 
cells in current weather depictions, but 
the forecast 4D trajectory was predicted 
to be clear of weather. 

A total of 144 unique trial scenarios were 
generated, with 36 trials in each of the 
encounter types. The 36 trials varied between 
the two distances conditions. 

2.4 Procedure 

Each trial began with a crosshair fixed at the 
center of a blank CSD display. After a variable 
amount of time, between 2-4 seconds, the trial 
display appeared with ownship in the center. 
The display range was preset to 160 nm (160 
nm in front and behind ownship), with the 
trajectory extending upward ahead of ownship 
to the edge of the display. Storm cells were 
located at a distance specified by the trial 


condition ahead of ownship in the upper half of 
the display. Pilots were asked to determine if a 
given flight trajectory was safe and, if the 
trajectory was determined to be unsafe, find a 
safe and efficient re-route around weather using 
the shortest amount of time. They were 
instructed to use their company’s standard 
operation procedure (SOP) for avoiding 
hazardous weather. Each trial ended when the 
pilot executed the modified flight trajectory 
using the RAT. On trials where pilots 
determined that no modification was necessary, 
they activated the RAT and executed the 
existing trajectory to terminate the trial. 
Following the end of a trial, a dialog box 
appeared in the center of the display with an OK 
button. As soon as the pilot clicked the OK 
button, the next trial began. 

The 144 trials were divided into three 
blocks, one for each weather viewing method 
(for details, see [23]). The order of the blocks 
was counterbalanced between pilots. Pilots 
received the corresponding training for a 
particular method right before that block of 
trials. 

On all trials the altitude of ownship was 
preset to 33000 feet; and the ground speed was 
preset to around 464 knots. Because it was 
assumed that the type of weather avoidance 
being studied here takes place en route, pilots 
were instructed to view the display in 2D and 
perform only lateral maneuvers even though 
CSD and RAT support 3D operations. No wind 
information was provided; pilots were instructed 
to infer wind direction based on the forecasted 
movement of the storm cells. 

3 Results 

The part-task study on which this paper is based 
was designed to evaluate usability and 
preference for three prototype methods for 
displaying predicted weather forecasts [23]. 
Results reported here are based on new analyses 
of the same set of data collapsed across the three 
methods which were assumed to not affect 
weather decisions differently. 

3.1 Evaluation Times 
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Evaluation times (i.e., elapsed time between the 
beginning of a trial and activating the RAT) 
provide a gross index of the difficulty in making 
deviation decisions. We have previously 
evaluated evaluation times according to 
encounter type and distance to weather [23]. 
The results showed that pilots spent slightly less 
time evaluating weather and proceeded quickly 
to modify the trajectory using the RAT when 
the distance to encounter was close (10.0 sec at 
40 nm and 10.8 sec at 80 nm). Pilots also spent 
more time on evaluation when the given 
trajectory was projected to go through the 
middle of a line of storms (11.3 sec) than when 
the given trajectory cleared the edge of the line 
of storms (9.2 sec). 

3.2 Deviation Decisions 

Deviation decisions were classified into one of 
four categories: no change, flying through gaps 
between storm cells, and flying around the 
leading or trailing edges. Fig. 2 plots the 
percentage distribution of deviation decisions by 
encounter type. As expected, different encounter 
scenarios elicited different mixtures of deviation 
decisions, and the particular mixtures reflected 
the opportunities afforded by the scenarios. For 
example, for scenario types with the initial 
trajectory flying through the middle of storms 
(middle and initially clearing gap), pilots mostly 
(about 80% time) chose to maneuver between 
gaps of storm cells as their initial course would 
have taken. In the remaining cases where pilots 
decided to fly around the storms, they mostly 
did so by flying around the trailing edge of the 


storms rather than the leading edge. 

The preference for pilots to fly around the 
trailing than leading edge of storms given 
choices is perfectly understandable: flying 
around the trailing edge means that the storm is 
moving away as the flight progresses, ffowever, 
this preference can be put aside when the initial 
trajectory is closer to the leading edge of a line 
of storms, such as the scenarios of the initially 
clearing edge encounter type. On those trials, 
pilots overwhelmingly (over 90% time) deviated 
for the weather by moving the trajectory further 
out from the leading edge of the storm. 

Trajectories in the clear later type of 
scenarios were designed so that if left 
unmodified they would have remained safe 
around weather. However, Fig. 1 shows that 
about 45% of time pilots still made 
modifications. We examined conditions under 
which pilots chose to modify the trajectories by 
analyzing their deviation decisions with respect 
to how the trajectories would have avoided 
weather. Among the 36 clear later trial, there 
were 24 trials with trajectories that would have 
avoided weather by flying through gaps, 1 trial 
with a trajectory that would have avoided 
weather by flying around the leading edge, and 
11 trials with trajectories that would have 
avoided weather by flying around the trailing 
edge. Fig. 3 shows the percentages of routing 
decisions as a function of how the trajectories 
would have avoided weather if left unchanged 
within each category. In general, for those 
trajectories that were modified, the distribution 
of deviation decisions seen in Fig. 2 reflected 
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Fig. 2. Percentage distributions of deviation 
decisions under different weather encounter types 


Fig. 3. Percentages of deviation decisions by the 
relation between initial trajectories and weather in 
the clear later type scenarios 
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the distribution of how the given trajectories 
would have avoided weather. 

3.3 Closest Point of Approach (CPA) to 
Weather 

While the types of encounter are expected to 
dictate deviation decisions to some extent, 
ideally they should not affect the maintenance 
of adequate safety margins from weather, which 
is 20 nm off any radar echo according to FAA 
guidelines [24]. We analyzed the distance 
between ownship trajectory and its closest point 
of approach (CPA) to weather for trials with 
middle and clear later types of encounters (Fig. 
4). These data show that, under both encounter 
types, the average CPAs varied significantly by 
deviation decisions, and in most cases fell 
below the FAA guidance; F(2,14) = 42.55, p 
<.0001 for middle type of encounters, F(3,21) = 
44.45, p<.0001 for clear later type of 
encounters. For the middle type of encounter 
scenarios, where “shooting the gap” was far and 
away the most efficient strategy, pilots mainly 
chose to fly through the gaps although this 
resulted in very small margins (~6 nm). 
Flowever, for the clear later type of scenarios 
where the original trajectories were already 
designed to clear weather by a margin around 6 
nm, pilots chose to modify the trajectories 40% 
of the time to increase the margin whenever 
possible. 



these results. The most significant one is that 
pilots do not always adhere to the FAA 
guidelines on the safety margin to be maintained 
from hazardous weather. It appears that pilots 
do make an effort in maintaining a safe distance 
from weather when the encounter circumstances 
make it easy to achieve, such as when the initial 
trajectories were at the boundaries of the leading 
or trailing edges of the storms. When the 
encounter circumstances make it inefficient to 
fly around the hazardous region altogether, such 
as when the initial trajectories take them 
through the middle of a line of storm cells, 
pilots would be more willing to fly closer to the 
storms to trade off comfort for efficiency. 

The current findings shed light on the 
observation by DeLaura and Evans where the 
pilot made a large deviation around weather 
conditions considered benign by CWAM [12], 
According to the CPA results shown in Fig. 4, 
what DeLaura and Evans described as “an 
unclear deviation strategy” appears to be what 
pilots do whenever possible, stretching their 
routes further away from storms when the other 
side of the airspace is completely clear. 

The current findings have practical 
implications for the development of automated 
weather avoidance algorithms. One suggestion 
for making the algorithms more tailored to pilot 
needs is to take into account opportunities 
present in the encounter circumstances, such as 
open airspace. Another suggestion is to 
incorporate a trade-off function between 
comfort and efficiency. In principle such a 
function could be derived from empirical 
observations of actual pilot decisions. 
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